Camera arrangement and method

ABSTRACT

A camera arrangement comprising a plurality of camera elements and a corresponding method of arranging camera elements is described. The camera arrangement improves the resolution of a stitched image by maintaining a uniform overlap between the fields of view of adjacent camera element. The uniform overlap is achieved. This is because the amount of unnecessary overlap required to achieve the required degree of overlap is reduced.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a camera arrangement and method.

2. Description of the Prior Art

A prior art device for producing a panoramic view from a number ofcameras exists. This is described in US-A-2006/0125921 and relates to avideo conferencing system. This device allows a group of individualco-located cameras to capture different fields of view of a scene. Theimages captured by these co-located cameras are then “stitched” togetherto form a panoramic image. In US-A-2006/0125921, the panoramic image isgenerated by warping and stitching together quadrilateral portions ofthe images from each camera.

Further, it is described in US-A-2006/0125921 that a section of thecomposite image can be extracted and displayed.

However, such a prior art device has very limited capabilities. Forexample, the device in US-A-2006/0125921 is only suitable for lowresolution panoramic images. This is because the techniques forextracting sections of the panoramic image described inUS-A-2006/0125921 will result in distortion to the extracted sections.

Additionally, with such a prior art device having low resolutionpanoramic images, the resolution of the extracted sections will belimited. This means that, with the ever increasing number of highdefinition devices being used by consumers, the use of the system in theprior art is particularly limited.

SUMMARY OF THE INVENTION

It is an aim and object of the present invention to address the problemsassociated with the prior art device.

According to one aspect of the present invention, there is provided acamera arrangement comprising a plurality of substantially co-locatedcamera elements, each camera element having a focal plane upon which arespective field of view is imparted, wherein the field of view of oneof the camera elements overlaps the field of view of at least oneadjacent camera element, wherein, after movement of the field of view ofone camera element, the positioning of said one camera element is suchthat said overlap between the field of view of said one camera elementand said at least one adjacent camera element is substantially uniformalong the length of the overlap. This is advantageous because the amountof unnecessary overlap between the fields of view, and thus the focalplanes, is reduced. This improves horizontal and vertical resolution ofthe correspondingly produced stitched image.

The said one camera element and/or said adjacent camera element in thearrangement may comprise a camera element adjuster operable to adjustany one or more of the pitch, yaw and roll positioning of said onecamera element and/or said adjacent camera element so that said overlapis substantially uniform along the length of the overlap.

Further, the roll of either said one camera element or said adjacentcamera element may be adjustable after said movement of said field ofview of said one camera element along the vertical axis of the field ofview.

The arrangement may comprise an arrangement adjuster operative to adjusteither one of the pitch and the yaw of the camera arrangement.

In this case, the arrangement may comprise a platform having a mountingand the platform may be operable to locate said one and at least onesaid adjacent camera elements thereon, the mounting being arranged toadjust either the pitch and the yaw of the camera arrangement.

The arrangement adjuster may be operable such that said pitch or yaw ofthe camera arrangement is adjustable when the overlap between the fieldof view is substantially constant along the length of the overlap.

At least one of the camera elements may be fixedly mounted to thearrangement adjuster.

According to another aspect, there is provided a method of arranging aplurality of co-located camera elements, comprising the steps of:

overlapping the field of view of one camera element with the field ofview of at least one other camera element, wherein the fields of vieware movable and imparted onto a focal plane of said respective cameraelements; and

positioning, after movement of said field of view of said one cameraelement, said one camera element such that said overlap between thefield of view of said one camera element and said at least one adjacentcamera element is substantially uniform along the length of the overlap.

The method may comprise adjusting any one of the pitch, yaw and rollpositioning of said one camera element and/or said adjacent cameraelement so that said overlap is substantially uniform along the lengthof the overlap.

The method may comprise adjusting said roll of either one of said onecamera element or said adjacent camera element after said movement ofsaid field of view of said one camera element along the vertical axis ofthe field of view.

The method may comprise adjusting either one of the pitch and the yaw ofthe camera arrangement.

The method may comprise mounting, on a platform, said one and at leastone of said adjacent camera elements and adjusting either the pitch andthe yaw of platform to adjust either the pitch and yaw of the cameraarrangement.

The method may comprise adjusting either one of said pitch or yaw of thecamera arrangement when the overlap between the field of view issubstantially constant along the length of the overlap.

The method may comprise mounting, on a platform said one and at leastone adjacent camera elements, and adjusting the yaw of said one cameraelement and/or said at least one adjacent camera element to achievedesired overlap.

The method may comprise fixedly mounting at least one camera element tothe platform.

BRIEF DESCRIPTION OF DRAWINGS

The above and other objects, features and advantages of the inventionwill be apparent from the following detailed description of illustrativeembodiments which is to be read in connection with the accompanyingdrawings, in which:

FIG. 1 is a schematic diagram showing a system for performing imagestitching containing a camera arrangement according embodiments of thepresent invention;

FIGS. 2A and 2B are diagrams showing two alternative cameraconfigurations in the system of FIG. 1;

FIG. 2C is a diagram showing the field of vision of the camera clustershown in FIG. 2B;

FIG. 3 is a diagram describing the stitching process implemented in theimage stitching means shown in FIG. 1;

FIG. 4 is a diagram describing the alignment process shown in FIG. 3;

FIGS. 5A-5D are diagrams describing the projection process shown in FIG.3;

FIGS. 6A and 6B are diagrams describing a process to generate a segmentfrom the image generated in the projection process of FIG. 5B;

FIG. 6C is a diagram describing outlining the segment generated in theprocess of FIGS. 6A and 6B on the image generated in the projectionprocess of FIG. 5B;

FIG. 7 is a diagram describing a storyboard running on the personaldevice as shown in FIG. 1;

FIG. 8 is a diagram describing a personal display device shown in thesystem of FIG. 1;

FIG. 9 is a diagram describing one arrangement of camera elements shownin FIG. 1;

FIG. 10 is a diagram describing the focal plane of the cameraarrangement shown in FIG. 9;

FIG. 11 is a diagram describing a camera arrangement according to oneembodiment of the present invention and the corresponding focal plane;

FIG. 12 is a diagram describing a second embodiment of the cameraarrangement of the present invention; and

FIG. 13 is a plan view of the platform shown in FIG. 12.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Referring to FIG. 1, the live event 101, which in this example is asoccer match is held in a venue, which in this example is a stadium.

A camera cluster or camera arrangement 102, which in this case consistsof six individual cameras 104 arranged in a certain configuration, ispositioned at an appropriate vantage point in the stadium. Theconfiguration of the camera cluster 102 will be explained in more detailwith reference to FIGS. 2A, 2B and 2C. However, in summary, the cameracluster 102 is configured so that the field of view of each camera 104within the camera cluster 102 overlaps to a small degree with the fieldof view of an adjacent camera 104 in the camera cluster 102. Thus, theentire live event is covered by panoramic view generated by the totalityof the field of view of the camera cluster 102. The vantage point may beat an elevated position in the stadium.

In one embodiment, each camera 104 is a High Definition (HD) camerawhose horizontal orientation is transformed by 90° so to produce aportrait image output having a resolution of 1080×1920 rather than1920×1080 as in the case of a traditional landscape orientation.Additionally, each camera 104 in this embodiment is operating inprogressive mode rather than interlaced mode. This makes processing ofthe images generated by the cameras 104 easier. However, the skilledperson will appreciate that each camera 104 may, alternatively, operatein interlaced mode. Using a number of these cameras 104 in a cameracluster 102 arranged in the portrait mode allows an output from thecamera cluster 102 to have a higher vertical picture resolution. Thecamera cluster 102 is used to produce a video stream of the soccermatch. As the skilled person would appreciate, although the cameracluster 102 is described as being composed of a number of individualcameras 104, the present invention is not so limited. Indeed, the cameracluster need not be made up of a concatenation of complete cameras 104,merely camera elements that each produce an image output. The cameracluster 102 may therefore be a single unit. Furthermore, it is possiblefor one single camera to be used to generate one output image. In thecase that the camera is used to capture a panoramic image, anappropriate wide angle lens and high resolution image capture arraywould be fitted to the camera.

In addition to the camera cluster 102, one or more microphones (notshown) may also be provided proximate the camera cluster 102 ordisparate to the camera cluster 102 to provide audio coverage of thesoccer match.

The output of each camera 104 in the camera cluster 102 is fed to achromatic aberration corrector 105. In this example, each camera 104within the camera cluster 102, produces an individual video output andso the camera cluster 102 has, in this case, six outputs. However, inother embodiments only one output of the camera cluster 102 may insteadbe used which is the multiplexed output of each of the six cameras 104.The output of the chromatic aberration corrector 105 is fed to an imagestitching means 108 and a scalable content preparation means 110 whichboth form part of an image processing device 106. The image processingdevice 106 consists of the image stitching means 108 and the scalablecontent preparation means 110 and in this embodiment, will be realisedon a computer. The output of the image stitching means 108 is connectedto the scalable content preparation means 110.

The image stitching means 108 takes each high definition image capturedby the respective camera 104 in the camera cluster 102 and combines themso as to produce a panoramic view of the venue. It is important to notethat in this embodiment, the output of the image stitching means 108 isnot simply the same view as taken using a wide angle lens. The output ofimage stitching means 108 is a tapestry, or conjoined, version of theoutput of each individual camera 104 in the camera cluster 102. Thismeans that the output of the image stitching means 108 has a resolutionof approximately 8000×2000 pixels rather than a resolution of 1080×1920pixels as would be the case if one HD camera was fitted with a wideangle lens. The conjoined image is therefore an ultra high resolutionimage. However, a single camera having a high resolution can be usedinstead. In this case, parts of the image stitching means 108 would notbe required. The advantages of the high definition arrangement arenumerous including the ability to highlight particular features of aplayer without having to optically zoom and therefore affecting theoverall image of the stadium. Further, the automatic tracking of anobject is facilitated because the background of the event is static andthere is a higher screen resolution of the object to be tracked. Theimage stitching means 108 is described in more detail with reference toFIG. 3.

The output of the image stitching means 108 is fed to either thescalable content preparation means 110 and/or one or more Super HighDefinition cinemas 128. In this embodiment, the or each super highdefinition cinema 128 is in a different location to the venue. Thisallows many spectators who are unable to attend the stadium due toshortage of capacity, or the location of the stadium, to view the liveevent. Additionally or alternatively, other locations around a stadiummay be used to situate the super high definition cinema 128. Forexample, a bar in the stadium serving refreshments may be used.

The scalable content preparation means 110 is used to generate an imagefrom the ultra high resolution output of the image stitching means 108so that it may be used by one or more High Definition televisions 120,personal display device 122 having a screen size smaller than atraditional television and/or the super high definition cinemas 124. Thescalable content preparation means 110 may generate either a scaled downversion of the ultra high resolution image or may generate a segment ofthe ultra high resolution image using the mapping technique explainedhereinafter. In one embodiment, the personal display device 122 is aPlayStation® Portable (PSP®). However, it is envisaged that the personaldisplay device 122 may also be a cell phone, laptop, Personal DigitalAssistant or the like or any combination thereof. Additionally, thescalable content preparation means 110 also implements an automatictracking algorithm to select parts of the ultra-high resolution image toproduce video streams for display on the personal display device 122.For example, the scalable content preparation means 110 mayautomatically track the ball or a particular player or even producefixed shots of a particular special event, such as scoring a goal in asoccer match or a touch-down in a US Football game.

The output of the scalable content preparation means 110 is fed to adistribution means 112. The distribution means 112 consists of a contentdatabase 114 that stores content which may be also distributed, forexample replays of special events, or further information relating to aparticular player etc. Also within the distribution means 112 is a datastreaming means 116 which converts the content to be distributed, eitherfrom the scalable content preparation means 110 or from the contentdatabase 114 into a format that has an appropriate bandwidth for thenetwork over which the streamed data is to be fed or broadcast. Forexample, the data streaming means 116 may compress the stream such thatit can be fed over an IEEE 802.11b WiFi network or over a cellulartelephone network or any appropriate network, such as a Bluetoothnetwork or a Wireless Network. In this embodiment, the network is a WiFinetwork which is appropriate for the personal display device 122 so theoutput of the data streaming means 110 is fed to a Wireless Router 118.Although the foregoing describes the data being fed over a WiFi networkor a cellular telephone phone network, the invention is not so limited.The data streaming means 116 may compress the stream for broadcast overany network which supports streaming video data such as a 3^(rd) or4^(th) generation cellular network, Digital Video Broadcast-Handheld(DVB-H) network, DAB network, T-DMB network, MediaFLO® network or thelike.

The super high definition cinema 124 includes a large screen projector126 and a screen 124. The output of the image stitching means 108 is fedto the large screen projector 126. In order to provide adequateresolution, the large screen projector 126 may have a display resolutionof 8000×2000 pixels or may consist of two conjoined projectors eachhaving a resolution of 4000×2000 pixels. Additionally, the large screenprojector 126 may include watermarking technology which embeds awatermark into the displayed image to prevent a user viewing the liveevent in the super high definition cinema 124 from making an illegalcopy of the event using a video camera. Watermarking technology is knownand will not be explained in any further detail.

Referring to FIG. 2A, the lenses of the cameras 104 in the cameracluster 102 are arranged in a horizontally convex manner. Alternatively,in FIG. 2B, the camera lenses of cameras 104 in the camera cluster 102are arranged in a horizontally concave manner. In either of the twoalternative configurations, the cameras 104 in the camera cluster 102are arranged to produce the minimum parallax effect between adjacentcameras 104 in the camera cluster 102. In other words, the cameras 104in the camera cluster 102 are arranged such that the focal point of apair of adjacent cameras are the closest together. The cameras 104 inthe arrangement of FIG. 2B have been found to produce a slightly lowerparallax error between adjacent cameras 104 than those of FIG. 2A.

In FIG. 2C, the field of view of the camera cluster 102 formed of fourcameras arranged in a horizontally concave manner is shown. This is forease of understanding and the skilled person would appreciate that anynumber of cameras can be used, including six as is the case with FIG. 1or two as is the case with FIGS. 6A and 6B and 9 to 13. As noted above,in order to ensure that the entire event is captured by the cameracluster 102, in embodiments of the present invention, the field of viewof one camera 104 in the camera cluster 102 slightly overlaps the fieldof view of another camera 104 in the camera cluster 102. This overlap isshown by the hashed area in FIG. 2C. As is explained hereinafter, theeffect of the overlap in the conjoined image is reduced in the imagestitching means 108. In the camera cluster 102 arranged in thehorizontally concave manner, the amount of overlap between the field ofview of different, adjacent, cameras 104 is substantially constantregardless of distance from the camera cluster 102. As the amount ofoverlap is substantially constant, the processing required to reduce theeffect of the overlap is reduced. Although the above is described withreference to arranging the cameras in a horizontal manner, the skilledperson will appreciate that the cameras may be arranged in a verticalmanner.

As described in relation to FIG. 1, the output from the camera cluster102 is fed into the chromatic aberration corrector 105. The chromaticaberration corrector 105 is known, but will be briefly described forcompleteness. The chromatic aberration error is corrected for eachcamera 104. The chromatic aberration manifests itself particularly atthe edge of images generated by each camera 104. As already noted, theimage output from each camera is 104 is stitched together. Therefore, inembodiments, the chromatic aberration is reduced by the chromaticaberration corrector 105 to improve the output ultra high resolutionimage.

The chromatic aberration corrector 105 separates the red, green and bluecomponents of the image from each camera 104 for individual processing.The red and green and blue and green components are compared to generatered and blue correction coefficients. Once the red and blue correctioncoefficients are generated, the red and blue corrected image componentsare generated in a known manner. The corrected red and blue imagecomponents are then combined with the original green image. This forms acorrected output for each camera 104 which is subsequently fed to theimage stitching means 108.

The image stitching means 108 then combines the aberration correctedindividual outputs from each camera 104 into the single ultra highdefinition image. The combining process is described with reference toFIG. 3.

The output from the chromatic aberration corrector 105 is fed into animage alignment means 301 and a virtual image projection means 304. Theoutput of the image alignment means 301 is fed a camera parametercalculation means 302. The output of the camera parameter calculationmeans 302 generates camera parameters which minimise the error in theoverlap region between two adjacent cameras 104. In this embodiment, theerror is the average mean squared error per pixel, although theinvention is not so limited. Also, only the roll, pitch, yaw, barrel andfocal length of each camera 104 are calculated. As the cameras 104 havesimilar focal lengths (the values of which are calculated) to reduce theparallax effect noted above and focal points, the relative positionbetween the cameras is not considered. It is envisaged that otherparameters may be found in order to allow for correction of lensdistortion, spherical aberrations, and the like. Additionally, it isnoted that chromatic aberration correction may again be performed afterthe alignment phase or after generation of the ultra high definitionimage.

The camera parameters are fed into the virtual image projection means304. The output of the virtual image projection means 304 is fed into acolour correction means 306. The output of the colour correction means306 is fed into an exposure correction means 308. The output of theexposure correction means 308 is fed into a parallax error correctionmeans 310. The output of the parallax error correction means 310 is thesingle ultra high definition image. As noted earlier, it is possible touse an image generated by one camera. In this case, the virtual imageprojection means 304 would not be required.

The image alignment means 301 is described with reference to FIG. 4. Itis to be noted that the following only describes finding the cameraparameters for two adjacent cameras. The skilled person will appreciatethat using this method, the camera parameters for any number of camerascan be found.

Live images A and B are generated by two respective adjacent cameras 104in the camera cluster 102. In order to minimise the error in the overlapregion, a hierarchical search technique is used by the image alignmentmeans 301. Using this method, it is assumed that the camera producingimage A is fixed. Both live images are fed into a low pass filter 402.This removes the fine details of the image. By removing the fine detailof the image, the likelihood of the search finding a local minimum isreduced. The amount of filtering applied to each image may be variedduring the search. For example, at the start of the search, a greateramount of filtering may be applied compared to at the end of a search.This means that an approximate value of the parameters may be generatedand may be refined towards the end of the search allowing a greateramount of detail to be considered and to improve the results.

The low pass filtered images are then fed into the virtual imageprojection means 304 shown in FIG. 3. The virtual image projection means304 is used to compensate for the fact that each camera 104 in thecamera cluster 102 is facing in a different direction but the ultra highresolution image to be generated should appear to come from one camerapointing in one direction. The virtual image projection means 304therefore maps one pixel of light received by one camera 104 onto avirtual focal plane. The virtual focal plane corresponds to the focalplane which would have been produced by a virtual camera capable ofcapturing the panoramic view with ultra high resolution. In other words,the output of the virtual camera would be the stitched ultra highresolution image. The manner in which the virtual image projection means304 operates is described in reference to FIGS. 5A-5D.

In order to generate the ultra high definition image suited for a curvedcinema screen, the virtual image projection means 304 maps each pixel oflight received by one camera 104 onto an appropriately curved virtualfocal plane. This is described with reference to FIGS. 5A and 5B.Alternatively, in order to generate the ultra high definition imagesuited for a flat cinema screen, the virtual image projection means 304maps each pixel of light received by one camera onto a flat virtualfocal plane. This is described with reference to FIGS. 5C and 5D.

Referring to FIG. 5A, the virtual image projection means 304 maps ontoeach pixel in the virtual curved focal plane 510, one pixel or aninterpolation of more than one pixel from the camera 104 in the cameracluster 102. The virtual curved focal plane 510 defines the focal planeof the virtual camera. The effect of this mapping is shown in FIG. 5A asimage 411.

The mapping according to this embodiment is described with reference toFIG. 5B. The mapped image on the virtual curved focal plane 510 isgenerated one pixel at a time. Therefore, the following will onlydescribe the mapping of one pixel onto a one-dimensional plane. However,the skilled person will be aware that in reality vector arithmetic willbe used to extend this idea onto a two-dimensional plane.

The dotted line 512 represents a light ray from an object at the liveevent. The light ray passes through the focal point 514 of the camera104 located in the camera cluster 102. The light ray then passes ontoone pixel in the image array 516 in the camera 104. In this case, thepixel is a Charge Couple Device (CCD) pixel in the camera 104. Thevirtual curved focal plane is determined to be a distance behind thepixel in the image array 516. Of course, the virtual focal plane may belocated in front of the image array 516. Additionally, it is possiblethat the virtual focal plane lies in a combination of in front of andbehind the image array 516. Therefore, knowing the camera parameters(yaw, pitch, roll, barrel distortion etc) which were calculated earlierin the alignment phase and the distance behind the image array 516 ofeach pixel in the virtual curved focal plane 510, the virtual imageprojection means 304 can determine for each pixel on the virtual curvedfocal point 510, which pixel (or interpolated value of more than onepixels) on the image array 516 should be mapped to the pixel on thevirtual curved focal plane 510. It should be noted that the pixel fromthe image array 516 may be filtered before being mapped to reducealiasing effects. However, such filtering is not essential.

Further, as already mentioned, one pixel on the virtual curved focalplane may correspond to one pixel on the image array 516 or acombination of more than one pixel on the image array 516. Inparticular, it may be that the pixel on the virtual curved focal pointmay be an interpolated value of more than one pixel on the image array516. This interpolated value may be an average value, a weighted averageor any other form of suitable interpolation which may depend on theproximity of the mapped pixel to one pixel on the image array 516compared with another pixel on the image array 516. As will also beexplained later, particularly in areas of overlap of the field of viewof two different cameras 104 in the camera cluster 102, a pixel on thecurved virtual focal plane 510 may be a pixel value or an interpolatedcombination of more than one pixel from image arrays 516 of differentalternative cameras.

Referring to FIG. 5C, the virtual image projection means 304 maps ontoeach pixel in the virtual flat focal plane 510′, one pixel or aninterpolation of more than one pixel from the camera 104 in the cameracluster. The virtual flat focal plane 510′ defines the focal plane ofthe virtual camera. The effect of this mapping is shown in FIG. 5C asimage 411′.

The mapping is described with reference to FIG. 5D. The mapped image onthe virtual flat focal plane is generated one pixel at a time.Therefore, the following will only describe the mapping of one pixelonto a one-dimensional plane. However, the skilled person will be awarethat in reality vector arithmetic will be used to extend this idea ontoa two-dimensional plane.

The dotted line 512′ represents a light ray from an object at the liveevent. The light ray passes through the focal point 514′ of the camera104 located in the camera cluster 102. The light ray then passes ontoone pixel in the image array 516′ in the camera 104. In this case, thepixel is a Charge Couple Device (CCD) pixel in the camera 104. Thevirtual flat focal plane is determined to be a distance behind the pixelin the image array 516′. Of course, the virtual focal plane may belocated in front of the image array 516. Additionally, it is possiblethat the virtual focal plane lies in a combination of in front of andbehind the image array 516. Therefore, knowing the camera parameters(yaw, pitch, roll, barrel distortion etc) which were calculated earlierin the alignment phase and the distance behind the image array 516′ ofeach pixel in the virtual flat focal plane, the virtual image projectionmeans 304 can determine for each pixel on the virtual flat focal plane510′, which pixel (or interpolated value of more than one pixels) on theimage array 516′ should be mapped to the pixel on the virtual flat focalpoint 510′. It should be noted that the pixel from the image array 510may be filtered before being mapped to reduce aliasing effects. However,such filtering is not essential.

Further, as already mentioned, one pixel on the virtual flat focal planemay correspond to one pixel on the image array 516′ or a combination ofmore than one pixels on the image array 516′. In particular, it may bethat the pixel on the virtual flat focal point may be an interpolatedvalue of more than one pixel on the image array 516′. This interpolatedvalue may be an average value, a weighted average or any other form ofsuitable interpolation which may depend on the proximity of the mappedpixel to one pixel on the image array 516′ compared with another pixelon the image array 516′. As will also be explained later, particularlyin areas of overlap of the field of view of two different cameras 104 inthe camera cluster 102, a pixel on the virtual flat focal plane 510′ maybe a pixel value or an interpolated combination of more than one pixelfrom image arrays 516′ of different alternative cameras.

When the ultra high resolution image is generated by being projectedonto a flat virtual focal plane, the aspect ratio appears, to the user,to be slightly incorrect. This is particularly apparent when viewing thewhole image. In order to reduce this effect, the aspect ratio of theflat virtual focal plane is adjusted. However, this results in thedivergence of vertical images. Therefore, in one embodiment, anappropriate aspect ratio which takes account of the above phenomena isdetermined.

Returning to FIG. 4, after the image has been mapped (resulting in ashot similar to that shown in 406), the mapped image is fed into anexposure corrector 408. The exposure corrector 408 is configured toanalyse the exposure and/or colourimetry of the overlap images producedby each camera 104 in the camera cluster 102. With this information, theexposure corrector 408 adjusts the exposure and/or colourimetryparameters of one camera to match those of the other camera.Alternatively, the exposure and/or colourimetry settings of one cameraare adjusted such that any sudden changes in exposure and/orcolourimetry are removed. However, it is possible that a combination ofthe above alternatives is utilised. It is advantageous to correct theexposure and/or colourimetry during the alignment process as thisresults in improved camera parameters. However, it is envisaged thatsuch parameters need not be corrected during the alignment process. Ifsuch parameters are not considered during alignment of the cameras, thensuch correction can be carried out on the images output from thecameras. In this case, it is to be noted that adjusting the image outputfrom one camera to match the exposure and/or colourimetry of the otherimage may increase the overall dynamic range of the image which wouldrequire additional storage and/or processing.

The image output from the exposure corrector 408 is fed into an errorgenerator 410 which is configured to determine the average mean squarederror per pixel in the overlap region for each set of chosen cameraparameters.

After the average mean squared error per pixel for one set of parametershas been generated, it is stored along with the camera parametersettings. Then, the camera parameters of the camera producing image Bare changed in an arbitrary manner and with arbitrary precision. Theaverage mean squared error per pixel for the changed parameters iscalculated and stored along with the camera parameter settings. Afterthe camera parameters of the camera producing image B have been changedacross a range, the camera setting with the lowest error are determined.The alignment process is then repeated using less low pass filtering andwith finer adjustments in the precision of the parameters. This processis repeated until the correct camera parameters, meeting the requirederror in the overlap, is generated. The camera parameters are thenstored within the image processing device 106, although the cameraparameters may be stored anywhere within the system.

It is noted that although the alignment process has been described withreference to live images, it is possible to use a calibration targetwhich is held in front of the camera. However, using this technique hasone distinct disadvantage. For a live event, the calibration target mayneed to be very large (in excess of 10 metres). Additionally, using liveimages means that if the camera(s) within the cluster move slightly, forexample, due to wind, small adjustments can be made in real-time withoutaffecting the live stream. For example, one of the previously storedminima could be used and the alignment process re-calibrated.Accordingly, the camera parameters may be determined “off-line” i.e. notlive on air, or “on-line” i.e. live on air if the re-calibration ofcameras is required.

Returning now to FIG. 3, the image stitching means 108 will be furtherdescribed. After the camera parameters have been established, the imageoutput from each camera is fed into a second image projection means 304.The output from the second image projection means 304 is fed into acolour corrector 306. The output from the colour corrector 308 is fedinto an exposure corrector 308. It is noted here that the functionalityof the second image projection means 304, the colour corrector 306 andthe exposure corrector 308 is the same as the image projector 404 andexposure and/or colourimetry corrector 408 described with reference toFIG. 4. This means that the ultra high definition image is subjected tothe same corrections as the individual images output from the cameras104.

The output of the exposure corrector 308 is fed into a parallax errorcorrector 310. The parallax error corrector 310 prevents “ghosting”which is caused when an object located in the overlap region of twocamera images appears twice when the images are stitched together.

In order to address this, in the stitched image, a mask is generated foreach of the overlap regions. It is then assumed that any significanterrors within the mask are caused by the parallax phenomenon. Theseerrors are quantified using the mean squared average error betweenpixels in the overlap region. This is a valid assumption as thealignment process minimised any errors due to camera parameters. Allindividual objects within the masks are labelled using knownmorphological and object segmentation algorithms. If the significanterror between pixels in the overlap region is below a threshold then thetwo images are blended together. Alternatively, in areas where the erroris high, ghosting is deemed to have taken place and only one image fromone camera is used. In order to reduce the parallax phenomenon, it isdesirable to have the focal points of each camera close together.

Referring to FIG. 1, the output of the image stitching means 108 is fedto the scalable content preparation means 110. As noted previously,various different streams of video are produced from the ultra-highresolution image output from the image stitching means 108. It isdesirable, as well as having manually guided or fixed images, thatautomatically generated segments of the ultra high resolution image areproduced. As is understood, the resolution of the segments of the ultrahigh resolution image is lower than that of the ultra high resolutionimage. This allows the segments to be multi-cast over a network to thepersonal display devices 122. Alternatively, or additionally, thesegments can be transmitted or broadcast to any device which supportsvideo streaming.

The generation of segments is explained with reference to FIGS. 5A, 5Band 6A. The processing will be carried out by the scalable contentpreparation means 110. Although FIG. 6A is described with reference togenerating a segment from the ultra high definition image produced bythe curved virtual focal plane, the segments of the ultra highdefinition image may be produced using the virtual flat focal plane, orindeed from a single camera output.

Referring to FIG. 6A, a virtual curved focal plane 510 showing apanoramic view of the venue 411 is generated as described with referenceto FIGS. 5A and 5B. The segment mimics the field of view of apseudo-camera, placed in the camera cluster 102, pointing in aparticular direction. In other words, the segment will appear to be thesection of the ultra high definition image if viewed from the focalpoint FP of the cameras 104 in the camera cluster 102 with the correctperspective. The segment is provided as the output from the focal plane602 of the pseudo camera. In FIG. 6A only the image array 516 of twocameras 104 is shown for clarity.

The geometry of the segment (i.e. the size and shape of the segment) isdetermined. It should be noted that the geometry of the segment (andaccordingly, focal plane 602 of the pseudo camera) can be varied and/orset depending on many parameters. For example, if the segment isgenerated to track a certain player on a soccer pitch, the geometry ofthe segment can be altered depending on the height of the player. Thismay be done by the user or automatically.

The segment of the ultra high definition image from curved focal plane510 to be viewed is then determined by a user or automatically. Oneexample of automatically determining the segment is using playertracking. In this example, the segment may be centred on the playerbeing tracked. The position of the segment is provided by a positionsignal. In order to assist a controller of the system, a box definingthe content of the segment may be provided on the displayed ultra highdefinition image. This box is useful for the cases where the segment ismanually or automatically determined. In the case of the automaticallygenerated segment, if the tracking algorithm stalls (i.e. fails to trackthe player between two or more successive frames), the controller maymanually assist the tracking algorithm to re-find the player. This boxmay be any shape and/or size and will be determined by the geometry ofthe segment as well as the geometry of the virtual focal plane 510. Thegeneration of the box will be explained hereinafter.

In order to generate the segment (and thus the focal plane 602 of thepseudo camera), the direction 602′ of the pseudo-camera is determinedrelative to the focal point FP of the cameras 104. More specifically,the direction 602′ between a location (which may be a point or smallnumber of pixels) in the ultra high definition image and the focal pointFP of the cameras 104 is determined. The direction 602′ of the pseudocamera relative to the focal point FP of the cameras 104 can bedetermined using the camera parameters calculated in the alignmentphase. More specifically, the location in the ultra high definitionimage is a certain direction from the focal point FP of the cameras 104which is calculated from the camera parameters determined earlier.

After the direction 602′ of the pseudo-camera is determined, the focalplane 602 of the pseudo camera is located substantially perpendicular tothe direction 602′. The focal plane 602 of the pseudo camera determinesthe size of the segment. The term substantially perpendicular covers notonly close to perpendicular, but also exactly perpendicular. In otherwords, the focal plane 602 of the pseudo camera will be substantiallyperpendicular to the focal point FP of the cameras 104 at that locationin the ultra high definition image.

In order to determine which pixels from the image array 516 are used onthe focal plane 602 of the pseudo camera, mapping is carried out in asimilar way to that described with reference to FIGS. 5B and 5D.Specifically, for each pixel in the focal plane 602 of the pseudocamera, the appropriate pixel or interpolated combination value of twoor more pixels in the image array 516 is determined using the cameraparameters generated in the alignment phase. The mapping takes place foreach pixel on the pseudo-camera focal plane 602.

As different segments are required during the course of the live event,for example as different players are tracked, as will be explainedhereinafter, this mapping of the pixels from the cameras 104 onto thefocal plane 602 of the pseudo-camera takes place for any number ofdifferent pseudo-camera positions and thus any number of differentpseudo-camera focal planes 602 and different directions 602′. This isshown in FIG. 6B which shows a different position of the pseudo-camera.Also, it should be noted that part of the focal plane of the pseudocamera 602 is in an overlap region. In this case, the mapping on thefocal plane 602 of the pseudo camera may require the selection of one ormore interpolated pixels from either of the image arrays 516 asexplained previously with reference to FIG. 3B. As is seen in FIG. 1,the scalable content means 110 can obtain the pixel information directlyfrom the output of the chromatic aberration corrector 105.

As noted earlier, the box containing the image which will form part ofthe segment may be generated. This box is shown in FIG. 1 and islabelled 415. As noted earlier, the location on the ultra highdefinition image of the segment is determined. This allows the direction602′ between the location on the ultra high resolution image and thefocal point FP of the cameras 104 to be determined using the cameraparameters. Additionally, as the geometry of the focal plane 602 of thepseudo-camera is known, it is possible to use vector geometry tocalculate where on the virtual focal plane 510 the outline of thesegment intersects with the image. This is shown in FIG. 6C.

As the scene 101 in FIG. 1 is described with reference to a flat virtualfocal plane, for consistency, a flat virtual focal plane is used in FIG.6C. Additionally, for clarity, the description in FIG. 6C is for aone-dimensional virtual focal plane and focal plane of the pseudocamera. The direction 602′ is determined and so the position of thepseudo-camera focal plane 602 is determined for the location in theultra high definition image. As the geometry of the focal plane 602 ofthe pseudo-camera has also been previously determined, the outline ofthe focal plane 602 of the pseudo camera is known. Therefore, usingvector geometry it is possible to calculate where on the virtual focalplane 510 (and therefore the ultra high definition image) the outline ofthe segment will lie. Thus, it is possible to outline on the ultra highdefinition image the segment which will be produced by the focal plane602 of the pseudo-camera. As is seen in FIG. 6C, the actual size andshape of the segment will change towards the edges of the ultra highdefinition image. This replicates the view of the pseudo-camera. Forexample, as seen in FIG. 6C, the segment will be slightly longer withthe pseudo-camera pointing in direction 602′ rather than if the pseudocamera was pointing in direction 606′. The slight increase in length isindicated as 604 in FIG. 6C.

In order to automatically generate the segments, the objects of interestwill need to be detected in the ultra high resolution image and trackedacross the image. The object detection is a probabilistic system basedon a combination of shape detection, image recognition andclassification technologies. Different kinds of detector are used fordifferent objects, for example a ball detector, player detector and thelike. Additionally, it is possible that a number of detectors of thesame kind will be used. For example, there may be 22 player detectors,each one trained to detect a different player. The detected object willthen be tracked across the ultra high resolution image. This allows auser to watch the segment of the ultra high resolution image in whichone particular player is located. This gives the effect of anautomatically generated “player cam”. The ultra high resolution imagereduces the error in the object detection and tracking algorithms.

In the event that the object tracking algorithm loses track of theplayer, the algorithm will identify the position on the ultra highresolution image where the player was last located. The object detectionalgorithm will then increase the area of search for the player aroundthis position. This will give the appearance of the image being “zoomedout”. Once the player is detected again, the tracking algorithm willthen continue tracking the player from the new location. In order toincrease the reliability of the player tracking algorithm, it isanticipated that more than one feature will be tracked. For example,both the player's head and shirt number may be tracked. Additionally, inorder to improve the viewer's experience, changes in direction of theviewed segments may be limited using temporal filters. This is to mimicmore closely the motion of a real television camera.

Each of the generated video streams are then fed to a distribution means112 for storing in the content database 114 and/or distribution of thecontent to spectators via the WiFi network 118, cell phone network (notshown) or any other appropriate network that supports video streamingvia the data streaming means 116.

Connected to the WiFi network 118 are a number of personal displaydevices 122. The personal display devices 122 are described withreference to FIG. 8. Each personal display device 122 has a screen 122c, navigation buttons 122 b, selection buttons 122 a and a networkadaptor 125 to connect the personal display device 122 to a network. Thenavigation buttons 122 b and the selection buttons 122 a allow the userto navigate between options available to view the video streams such aswhich video stream to view and the like. Also provided in the displaydevice 122 is a memory 123 which contains a suitable program to processthe received video streams. Such a program may also include a decryptionalgorithm to decrypt encrypted video streams. The skilled person wouldappreciate that the program may also be implemented in hardware insteador in addition to the program. In order to ensure that each personaldisplay device 122 can view the video streams, the data streaming means116 compresses the video stream using the MPEG-4 format. However, it isanticipated that any other compression format is appropriate to allow areasonable quality video stream to be transmitted at a low bit rate. Inthe case of MPEG-4, each video stream will have a bit rate ofapproximately 500 Kb/s.

In order to allow a scalable number of spectators to view the videostreams, a multi-cast connection is set-up between the data distributionmeans 116 and the personal display devices 122. When using the WiFiconnection, a Multi-cast address, within the Multi-cast address range,is determined for each video stream. Each personal display device 122 isconfigured to receive such multi-cast video streams. Further, in orderto allow the spectator the opportunity of selecting the most appropriatevideo stream, a thumbnail version of each video stream will be providedon the personal display device 122 from which the user can select themost appropriate video stream.

In embodiments, during the match, a buffer of 15 seconds for each videostream is recorded in the personal display device 122. This means thatshould a particular event happen such as a goal is scored or a player issent off, the personal display device 122 will store that 15 seconds offootage and will automatically display these as highlights for instantplay-back or play-back at the end of the match. The personal displaydevice 122 will be notified of such an event by the data distributionmeans 116. Also, the content database 114 will also store such footagefor transmission after or during the match as a separate video stream.

In FIG. 7, a storyboard explaining a sequence of events on the personaldisplay device 122 is shown. In this storyboard, a user is attending asoccer match between two fictitious soccer teams, Thames Valley FootballClub and London City Football Club at Thames Valley Football Club'sground. The application starts to run on the user's personal displaydevice 122. This shows the London City Football Club logo (the user is aLondon City Football Club supporter). The user enters an access code toenable connection to the server. The access code is provided to the userbefore the event either when the user rents the personal display deviceor when they obtain their season ticket. The personal display deviceconnects to the data streaming means 116. The user is then provided witha number of options. In screen 4 of the storyboard, the “home page” isshown. This gives details of the soccer match. Using the navigationbuttons 122 b on the personal display device 122, the users selects oneof the four options. In screen 5, the user has requested details of theLondon City Football Club team. In screen 6, the user has returned tothe “home page” which shows that London City Football Club has scored agoal. The return to the home page may be done automatically underinstruction from the data streaming mean 116 or manually by the user. Inscreen 7, the user is watching the live stream provided by the datastream and in screen 8 the user is watching a highlights stream.

In order to provide this information to the user, the personal displaydevice 122 has access to three data streams. The first is the livestream which is provided by the scaleable content means 110, the secondis the highlights stream provided by the content database 114 (thehighlights stream may provide highlights from this match or previousmatches) and the third is a data stream.

The personal display device 122 “listens” to the data stream and storesand updates stored information onto the memory 123. The data stream maycontain information such as team news, goals, substitutes, latest score,goal scorers and match time. Also, the data channel may contain imagedata, such as highlights of video. However, in embodiments, anyancillary information (sometimes referred to as metadata) may beprovided. Such metadata may include information relating to the ultrahigh definition image or any segments of the ultra high definition imagesuch as good shot markers which may be used by the personal displaydevice 122 to store the appropriate images in the buffer of the memory123 or may relate to the content of the ultra high definition image orany segments of the ultra high definition image such as a flag when agoal is scored. The personal display device 122 may listen continuallyor periodically to the data stream. In one embodiment, the data streamwould be provided in a data carousel manner using an eXtended MarkupLanguage (XML) file, although other approaches are possible as would beappreciated. Indeed, any suitable technique, such as a digital fountaincould be used that provides a good chance of receiving data without useof a back channel.

Each video stream and the data stream will be encrypted. This means, inorder to be able to access the video stream on their personal displaydevice 122, a user will have to pay in order to receive a decryptionkey. Additionally, the users will have flexibility to choose the numberof video streams they receive. For example, the user may only select toaccess the highlight video from the content database 114. Alternatively,the user may pay a larger premium and access all video streams. It isexpected that the user would receive these decryption keys before theevent after registering the personal display device 122 on a dedicatedwebsite. The decryption keys would then be provided. An alternative oradditional way in which the user can be provided with the decryption keyis by selecting the level of service they require when arriving at thevenue. The decryption keys will then be provided over the WiFi networkor by other means for example by providing a Flash memory card with thedecryption keys loaded thereon. The video streams may be provided on agame by game basis or season by season. It is also possible that thevenue will rent out the personal display devices 122 to spectators aswell as providing access to the video streams.

Additionally, as noted in FIG. 1, the scalable content means 110 alsofeeds a video stream to a High Definition Television display 120. Thiswill be provided on a High Definition Serial Digital Interface (HDSDI)output to be connected to a broadcaster's video switch or to an AudioVideo Channel (AVC) codec for direct transmission. The stream to theHigh Definition Television display 120 will be generated using a balltracking algorithm although manual control will be possible. Further, itis anticipated that the High Definition Television display 120 may alsodisplay the whole pitch image. However, due to the size of the image,this would require considerable Letter Box effect. It is alsoanticipated that the segments of video stream can be fed to a pair ofvirtual reality goggles 122 a. In this case, the direction of where thegoggles are facing will allow an appropriate video segment to begenerated.

The super high definition cinema 124 is located away from the venue.This increases the effective capacity of the venue as well as providingan increase in revenue from refreshments and the like. It is expectedalso, that the super high definition cinemas 124 may be located in adifferent country to the event. In this case, the stitched image may betransmitted by satellite. This allows world-wide coverage of an event,for example the soccer World Cup or International cricket matches andthe like or concerts such as Live-8 and the like.

The skilled person would appreciate that the processing of the imagesignals produced by each camera 104 in the camera cluster 102 to producethe ultra-high definition image and the viewing of the video streams onthe personal display device 122 may be carried out by at least onemicro-processor running a computer program. The computer program will beprovided on a storage medium which may be a magnetically or opticallyreadable medium or indeed as a signal provided over a network such as alocal area network, a wide area network or the Internet.

Although the foregoing has been described with reference to live events,the skilled person would appreciate that the system may be used in anynumber of other situations. For example, the present invention may beuseful in a surveillance system where tracking of criminal suspects overa wide area is important. As noted above, the present invention isparticularly suited to automatic object (including face) detection andtracking because of the high resolution of the image.

Although the foregoing has described the segments of the ultra highdefinition image as being sent to personal display devices 122, thesegments may alternatively, or additionally, be sent to mobiletelephones or may be included as a “picture-in-picture” type arrangementin the ultra high definition image. Further, the segment may begenerated on recorded ultra-high definition images so that segments ofhighlights footage can be generated. In this case, metadata may beattributed to the footage such as the camera parameters to allow thesegments to be generated.

Although the foregoing has been described with reference to mapping thesegment using pixels from the image array of the camera(s) 104, it willbe understood that should the images from the camera 104 not beavailable, it is possible to map the segment using pixel informationfrom the ultra high definition image. In this case, it is useful to havean estimate of one camera parameter such as focal length provided withthe image data. However, obtaining segment from the ultra highdefinition image may reduce the resolution of the segment because theultra high definition may be composed of interpolated pixel values.Therefore, when the ultra high resolution image is used, the resolutionof the generated segment may be reduced.

Turning now to FIG. 9, as noted above, in order to generate thesuper-high definition image the camera arrangement 102 is used tocapture the event. Each camera element in the camera arrangement 102 hasdifferent, but overlapping, fields of view. The camera elements arelocated near to one another. As noted above, the lens of one cameraelement should be placed as close to a lens of an adjacent cameraelement as possible without actually touching. This reduces parallaxeffects and ensures that the lens of one camera element is not capturedin the field of view of the other camera element. The captured imagesare then stitched together to form the ultra-high definition image. FIG.9 describes one way of producing the camera arrangement 102 having twoco-located camera elements 1004 and 1006.

In part (a) of FIG. 9, in embodiments of the present invention eachcamera element 1004 and 1006 is a high definition camera mounted inlandscape mode on an individual camera stand 1010. As shown in part (a)of FIG. 9, each camera element has a lens initially directed out of thepage. Each camera stand 1010 has a mounting bracket 1012 upon which therespective camera element 1004 and 1006 is mounted. The mounting bracket1012 secures the camera element 1004 and 1006 to legs 1014 whichsupport, and elevate, the camera element.

The mounting bracket 1012 allows the camera element which is mountedthereon to be controlled using pitch and yaw. Pitch is a conventionalterm meaning bidirectional movement in the vertical plane (the y-axis inFIG. 9) and yaw is a conventional term meaning bidirectional movement inthe horizontal plane (the x-axis in FIG. 9).

As noted earlier, the individual camera elements are used in the cameraarrangement to give the effect of a single camera with a very largefield of view. In fact, the overall field of view of the cameraarrangement 102 is the sum of the field of view of each camera element1004 and 1006 subtract the region in which the field of view of eachcamera element 1004 and 1006 overlap. For example, if the cameraarrangement 102 has two camera elements 1004 and 1006, each having afield of view of 60° and having a 5° overlap, the effective overallfield of view of the camera arrangement 102 is 115°.

In order to produce such an overall field of view, the yaw of eachcamera element 1004 and 1006 is adjusted so that the camera lens pointsin an appropriate direction. To reduce parallax effects it is useful tohave similar focal points for each camera element. Therefore, it isuseful to locate the lens of each camera element close to one another.Such an arrangement is shown in part (b) of FIG. 9.

In many situations, such as capturing a sporting event, the cameraarrangement 102 will be located in a position that is elevated comparedwith the event being captured. This means that in order to capture theevent, the pitch of each camera element 1004 and 1006 will be adjusted.In particular, as shown in part (c) of FIG. 9, each camera element 1004and 1006 will be adjusted so that the camera lens is directed down andtowards the event to be captured.

In FIG. 10, the focal plane 5160 of camera element 1004 and the focalplane 5162 of camera element 1006 is shown. Additionally, the area ofoverlap 5168 required to enable the image captured by each cameraelement 1004 and 1006 to be stitched together is shown. The area ofoverlap 5168 is highlighted with dots in FIG. 10. The resultant ultrahigh definition image 124 generated by the stitching is also shown. Thearea of overlap 5168 is required to stitch the images captured by eachcamera element 1004 and 1006 together. The size of the area of overlap5168 will vary depending on the event to be captured. For example, anevent with a large amount of detail would require a smaller area ofoverlap than an event with a smaller level of detail. This would beappreciated by the skilled person. The area of overlap 5168 is ofarbitrary size “d” in FIG. 10 because the required area of overlapdepends on the event to be captured. In the present situation, where asoccer match is being captured, the area of overlap, d, will be around5°. It is noted here however that different situations may demand largeror smaller overlaps.

As the overlap of area d is required, the focal planes 5160 and 5162need to overlap sufficiently to achieve this minimum overlap area. Asillustrated in FIG. 10, because of this, there is an area of additional,unnecessary overlap 5166 defined by the cross-hatched lines. Thisunnecessary overlap 5166 results in an overall reduction in horizontalresolution of the ultra high definition image. Moreover, as shown by thehatched lines, there is an area 5164 above and below the super highdefinition image 124 which does not form part of the stitched ultra highdefinition image. This unnecessary area 5164 results in a reducedvertical resolution ultra high definition image.

In order to improve the horizontal and vertical resolution of the ultraor super high definition image, one or each of the camera elements 1004and 1006 in an embodiment of the present invention is mounted on adifferent type of mounting bracket 1012′. Similarly to the mountingbracket 1012 in FIG. 9, the mounting bracket 1012′ of this embodimenthas pitch and yaw control. However, additionally, the mounting bracket1012′ of this embodiment allows the respective camera element 1004 and1006 to roll. Roll is a conventional term meaning rotational motionaround a central axis substantially through the centre axis of the lens.

Thus, after the camera elements 1004 and 1006 have been moved to theposition explained with reference to FIG. 9, the roll of one or eachcamera element 1004 and 1006 is adjusted. In the case of the cameraelements of FIG. 11, camera element 1004 is rolled anti-clockwise(counter-clockwise) when looking through the lens of the camera element1004 and camera element 1006 is rolled clockwise when looking throughthe lens of the camera element 1006. It is noted that, in this case bothcamera elements 1004 and 1006 are rolled. However, this is notnecessarily always the case. It may be that only one camera element 1004and 1006 need to be rolled.

The result of this roll is that one edge of the focal plane of cameraelement 5160 and one edge of the focal plane of camera element 5162 aresubstantially parallel to each other. The area of overlap 5168 is stilld as is required to enable stitching of the images captured by cameraelement 1004 and camera element 1006. However, as is illustrated in FIG.11, the proportion of the focal plane 5160 of camera element 1004 and ofthe focal plane 5162 of camera element 1006 which is used to generatethe super high definition image 124 is increased. Accordingly, the area5164 defined by the hatched lines is reduced and so improving both thevertical and horizontal resolution of the super high definition image.

Moreover, as one edge of the focal plane 5160 of camera element 1004 andone edge of the focal plane 5162 of camera element 1006 are adjusted tobe substantially parallel to one another, the yaw of camera elements1004 and 1006 are adjusted to maximise the overall field of view of thecamera arrangement 102 whilst still providing at least the minimumoverlap area required to perform image stitching. Thus, by using thisembodiment of the present invention, the horizontal resolution of thesuper high definition image is improved.

However, mounting bracket 1012′ which includes the roll facility is notas common as mounting bracket 1012 which does not include such afacility. Accordingly, the cost of mounting bracket 1012′ is higher thanthe cost of mounting bracket 1012. Additionally, by applying roll to thecamera element 1004 and/or 1006 a load imbalance is applied to thecamera support. Due to the weight of high definition camera elements1004 and 1006 the load imbalance on each camera support will be notinsignificant.

In order to improve this embodiment, a further embodiment of the presentinvention will now be described with reference to FIG. 12.

Camera elements 1004 and 1006 are mounted on respective third mountingbrackets 1208. The third mounting brackets 1208 need only be capable ofyaw movement, although the third mounting brackets 1208 may also becapable of pitch movement. Although not preferable, the third mountingbracket 1208 may include roll movement, but as explained above, amounting bracket including roll movement is more expensive and tends tobe the subject of load imbalance when a camera element 1004 and 1006 ismounted thereon.

The third mounting brackets 1208 are located on a camera platform 1200.The third mounting brackets 1208 are mounted on the platform 1200 sothat the third mounting brackets 1208 may move bi-directionally alongthe platform 1200. Camera platform 1200 is mounted on a platform bracket1202 which allows yaw and pitch movement. The yaw and pitch movement iscontrolled by control arm 1206. The platform bracket 1202 is mountedonto a support 1204 which may be a table top stand (as shown) or legs.The support 1204 is used to support the weight of the cameraarrangement.

Referring to FIG. 13, the mounting brackets 1208 engage with respectiverunners 1210. The runners 1210 may be located on the surface of theplatform 1200 or may be located below the surface of the platform 1200.Also, although one individual runner per mounting bracket is shown, theskilled person will appreciate that any number of individual runners maybe equally used. Indeed, only a single runner may be provided alongwhich the mounting brackets 1208 may move. In order to engage with therespective runner 1210, the mounting bracket 1208 must be suitablyshaped. The runners 1210 allow the respective mounting bracket 1208 tobe moved therealong. From the plan view of the platform 1200 shown inFIG. 13, a screw fitting 1212 is shown on the mounting bracket 1208which engages the respective camera element 1004 and 1006 to themounting bracket 1208. Other types of fitting may also or instead beused as would be appreciated.

Although not shown, each mounting bracket 1208 also includes a lockingmechanism which, when activated, locks the mounting bracket 1208 in afixed position along the runner 1210. When the locking mechanism is notactivated, the mounting bracket 1208 is free to move along the runner1210.

Referring back to FIG. 12, in use, each respective camera element 1004and 1006 is mounted onto a mounting bracket 1208. The respective cameraelement 1004 and 1006 engages with the screw fitting 1212. The cameraelement 1004 and 1006 are arranged so that the pitch of each cameraelement is the same, which may be for example 0° with respect to thehorizontal axis of the field of view. The yaw of each camera element1004 and 1006 is adjusted to obtain the correct field of view of theevent to be captured. Additionally, the yaw is adjusted to ensure thatthere is appropriate overlap between each field of view. In thisembodiment, the yaw of each camera element 1004 and 1006 is adjusted toachieve an overlap of 5°.

The locking mechanism on each mounting bracket 1208 is deactivated (ifit was previously activated) so that the mounting brackets 1208, andconsequently the respective camera elements 1004 and 1006, can movealong the runner 1210. The mounting brackets 1208 are moved along therunner 1210 to reduce the distance between each camera element 1004 and1006. In other words, the camera elements 1004 and 1006 are movedtowards each other. Each camera element 1004 and 1006 are moved togetherso that the lens of each camera element 1004 and 1006 is as closetogether as possible without touching. This is to ensure that the lensof one camera element does not appear in the field of view of the othercamera element. Additionally, by arranging the camera elements to beclose together allows the focal point of each camera element 1004 and1006 to be very similar. As noted above, this reduces parallax effects.However, this procedure may alter the area of overlap. If this doeshappen, then the yaw of any one or more of the camera elements isadjusted to maintain the minimum overlap.

When a suitable distance apart, and with the suitable overlap, thelocking mechanism is activated to fix the position of the mountingbracket 1208 along the runner 1210. By locking the camera elements 1004and 1006 in position in this manner, the edges of each focal plane ofeach camera element 1004 and 1006 are substantially parallel to oneanother, with the required overlap. This results in the focal planearrangement shown in FIG. 11.

Once the camera elements 1004 and 1006 are fixed in position, the pitchand yaw of the camera platform 1200 can be adjusted by the controlhandle 1206. This allows the focal plane arrangement to be moved withoutthe individual focal plane of either camera element 1004 and 1006 beingmoved relative to the other camera element 1006 and 1004. Consequently,the pitch and yaw of the focal plane arrangement of FIG. 11 is adjustedas a whole meaning that the advantageous focal plane arrangement of FIG.11 is not affected by adjusting the pitch and yaw of the camera platform1200.

It is anticipated, in one embodiment, that the yaw of the cameraelements 1004 and 1006 on the camera platform 1200 will be adjusted tothe correct orientation before the camera platform 1200 is located inthe stadium. However, the invention is not so limited. Further, althoughthe forgoing has been described with a separate runner for each cameraelement, the invention is not so limited. Indeed, in one embodiment itis anticipated that the camera platform 1200 would have one runner andeach camera element 1004 and 1006 would be mounted on a common runner.This provides an additional advantage of having a generic platformallowing any number of camera elements to be located thereon.Additionally, although the forgoing has been described with a cameraarrangement containing only two camera elements, the invention is not solimited. For example, the camera arrangement may have any number ofcamera elements, such as six or eight camera elements. Additionally, thecamera elements have been described as being positioned in the landscapemode. However, the camera elements may be positioned in the portraitmode. This is useful to increase vertical resolution and as would beappreciated, would be particularly applicable when the cameraarrangement consists of more than two camera elements, such as sixcamera elements.

Although illustrative embodiments of the invention have been describedin detail herein with reference to the accompanying drawings, it is tobe understood that the invention is not limited to those preciseembodiments, and that various changes and modifications can be effectedtherein by one skilled in the art without departing from the scope andspirit of the invention as defined in the appended claims.

1. A camera arrangement comprising a plurality of substantiallyco-located camera elements, each camera element having a focal planeupon which a respective field of view is imparted, wherein the field ofview of one of said camera elements overlaps the field of view of atleast one adjacent camera element, wherein, after movement of the fieldof view of one camera element, the positioning of said one cameraelement is such that said overlap between the field of view of said onecamera element and said at least one adjacent camera element issubstantially uniform along the length of the overlap.
 2. A cameraarrangement according to claim 1, wherein said one camera element and/orsaid adjacent camera element comprises a camera element adjusteroperable to adjust any one or more of the pitch, yaw and rollpositioning of said one camera element and/or said adjacent cameraelement so that said overlap is substantially uniform along the lengthof the overlap.
 3. A camera arrangement according to claim 1, whereinsaid roll of either said one camera element or said adjacent cameraelement is adjustable after said movement of said field of view of saidone camera element along the vertical axis of the field of view.
 4. Acamera arrangement according to claim 1, comprising an arrangementadjuster operative to adjust either one of the pitch and the yaw of thecamera arrangement.
 5. A camera arrangement according to claim 4,comprising a platform having a mounting, wherein the platform isoperable to locate said one and at least one said adjacent cameraelements thereon, and the mounting is arranged to adjust either thepitch and the yaw of the camera arrangement.
 6. A camera arrangementaccording to claim 4, wherein said arrangement adjuster is operable suchthat said pitch or yaw of the camera arrangement is adjustable when theoverlap between the field of view is substantially constant along thelength of the overlap.
 7. A camera arrangement according to claim 4,wherein at least one of the camera elements is fixedly mounted to thearrangement adjuster.
 8. A method of arranging a plurality of co-locatedcamera elements, comprising the steps of: overlapping the field of viewof one camera element with the field of view of at least one othercamera element, wherein the fields of view are movable and imparted ontoa focal plane of said respective camera elements; and positioning, aftermovement of said field of view of said one camera element, said onecamera element such that said overlap between the field of view of saidone camera element and said at least one adjacent camera element issubstantially uniform along the length of the overlap.
 9. A methodaccording to claim 8, comprising adjusting any one of the pitch, yaw androll positioning of said one camera element and/or said adjacent cameraelement so that said overlap is substantially uniform along the lengthof the overlap.
 10. A method according to claim 8, comprising adjustingsaid roll of either one of said one camera element or said adjacentcamera element after said movement of said field of view of said onecamera element along the vertical axis of the field of view.
 11. Amethod according to claim 8, comprising adjusting either one of thepitch and the yaw of the camera arrangement.
 12. A method according toclaim 11, comprising mounting, on a platform, said one and at least oneof said adjacent camera elements and adjusting either the pitch and theyaw of platform to adjust either the pitch and yaw of the cameraarrangement.
 13. A method according to claim 11, comprising adjustingeither one of said pitch or yaw of the camera arrangement when saidoverlap between the field of view is substantially constant along thelength of the overlap.
 14. A method according to claim 11, comprisingmounting, on a platform said one and at least one adjacent cameraelements, and adjusting the yaw of said one camera element and/or saidat least one adjacent camera element to achieve desired overlap.
 15. Amethod according to claim 11, comprising fixedly mounting at least onecamera element to said platform.
 16. A camera arrangement comprising aplurality of substantially co-located camera element means, each cameraelement means having a focal plane upon which a respective field of viewis imparted, wherein the field of view of one of said camera elementsmeans overlaps the field of view of at least one adjacent camera elementmeans, wherein, after movement of the field of view of one cameraelement means, the positioning of said one camera element means is suchthat said overlap between the field of view of said one camera elementmeans and said at least one adjacent camera element means issubstantially uniform along the length of the overlap.